Add a python JIT loader class. #142514
Add a python JIT loader class. #142514
Conversation
Users can implement a JIT loader in python by implementing a class that named "JITLoaderPlugin" in a module. The path to this module is specified in the settings using:
(lldb) setting set target.process.python-jit-loader-path <path>
When the process starts up it will load this module and call methods on the python class. The python class must implement the following methods:
```
class JITLoaderPlugin:
#-----------------------------------------------------------------------
# Construct this object with reference to the process that owns this
# JIT loader.
#-----------------------------------------------------------------------
def __init__(self, process: lldb.SBProcess):
self.process = process
#-----------------------------------------------------------------------
# Called when attaching is completed.
#-----------------------------------------------------------------------
def did_attach(self):
pass
#-----------------------------------------------------------------------
# Called when launching is completed.
#-----------------------------------------------------------------------
def did_launch(self):
pass
#-----------------------------------------------------------------------
# Called once for each module that is loaded into the debug sessions.
# This allows clients to search to symbols or references to JIT'ed
# functions in each module as it gets loaded. Note that this function
# can be called prior to did_attach() or did_launch() being called as
# libraries get loaded during the attach or launch.
#-----------------------------------------------------------------------
def module_did_load(self, module: lldb.SBModule):
pass
```
|
@llvm/pr-subscribers-lldb Author: Greg Clayton (clayborg) ChangesUsers can implement a JIT loader in python by implementing a class that named "JITLoaderPlugin" in a module. The path to this module is specified in the settings using: When the process starts up it will load this module and call methods on the python class. The python class must implement the following methods: Patch is 40.07 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/142514.diff 28 Files Affected:
diff --git a/lldb/bindings/python/python-wrapper.swig b/lldb/bindings/python/python-wrapper.swig
index 3d1d04e47e70b..9e8528357bf76 100644
--- a/lldb/bindings/python/python-wrapper.swig
+++ b/lldb/bindings/python/python-wrapper.swig
@@ -542,6 +542,18 @@ void *lldb_private::python::LLDBSWIGPython_CastPyObjectToSBEvent(PyObject * data
return sb_ptr;
}
+void *lldb_private::python::LLDBSWIGPython_CastPyObjectToSBModule(PyObject * data) {
+ lldb::SBModule *sb_ptr = nullptr;
+
+ int valid_cast =
+ SWIG_ConvertPtr(data, (void **)&sb_ptr, SWIGTYPE_p_lldb__SBModule, 0);
+
+ if (valid_cast == -1)
+ return NULL;
+
+ return sb_ptr;
+}
+
void *lldb_private::python::LLDBSWIGPython_CastPyObjectToSBStream(PyObject * data) {
lldb::SBStream *sb_ptr = nullptr;
diff --git a/lldb/include/lldb/API/SBModule.h b/lldb/include/lldb/API/SBModule.h
index 85332066ee687..7f4e9ef470532 100644
--- a/lldb/include/lldb/API/SBModule.h
+++ b/lldb/include/lldb/API/SBModule.h
@@ -306,6 +306,7 @@ class LLDB_API SBModule {
friend class SBType;
friend class lldb_private::python::SWIGBridge;
+ friend class lldb_private::ScriptInterpreter;
explicit SBModule(const lldb::ModuleSP &module_sp);
diff --git a/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h b/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h
new file mode 100644
index 0000000000000..b18d7ae5ae3f5
--- /dev/null
+++ b/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h
@@ -0,0 +1,30 @@
+//===-- JITLoaderInterface.h ------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_INTERPRETER_INTERFACES_JITLOADERINTERFACE_H
+#define LLDB_INTERPRETER_INTERFACES_JITLOADERINTERFACE_H
+
+#include "ScriptedThreadInterface.h"
+#include "lldb/lldb-private.h"
+
+namespace lldb_private {
+class JITLoaderInterface : virtual public ScriptedInterface {
+public:
+
+ virtual llvm::Expected<StructuredData::GenericSP>
+ CreatePluginObject(llvm::StringRef class_name,
+ lldb_private::ExecutionContext &exe_ctx) = 0;
+
+ virtual void DidAttach() {};
+ virtual void DidLaunch() {};
+ virtual void ModulesDidLoad(lldb_private::ModuleList &module_list) {};
+
+};
+} // namespace lldb_private
+
+#endif // LLDB_INTERPRETER_INTERFACES_JITLOADERINTERFACE_H
diff --git a/lldb/include/lldb/Interpreter/ScriptInterpreter.h b/lldb/include/lldb/Interpreter/ScriptInterpreter.h
index 3a4a7ae924584..99531844c80f3 100644
--- a/lldb/include/lldb/Interpreter/ScriptInterpreter.h
+++ b/lldb/include/lldb/Interpreter/ScriptInterpreter.h
@@ -16,6 +16,7 @@
#include "lldb/API/SBEvent.h"
#include "lldb/API/SBExecutionContext.h"
#include "lldb/API/SBLaunchInfo.h"
+#include "lldb/API/SBModule.h"
#include "lldb/API/SBMemoryRegionInfo.h"
#include "lldb/API/SBStream.h"
#include "lldb/Breakpoint/BreakpointOptions.h"
@@ -24,6 +25,7 @@
#include "lldb/Core/ThreadedCommunication.h"
#include "lldb/Host/PseudoTerminal.h"
#include "lldb/Host/StreamFile.h"
+#include "lldb/Interpreter/Interfaces/JITLoaderInterface.h"
#include "lldb/Interpreter/Interfaces/OperatingSystemInterface.h"
#include "lldb/Interpreter/Interfaces/ScriptedPlatformInterface.h"
#include "lldb/Interpreter/Interfaces/ScriptedProcessInterface.h"
@@ -560,6 +562,10 @@ class ScriptInterpreter : public PluginInterface {
return {};
}
+ virtual lldb::JITLoaderInterfaceSP CreateJITLoaderInterface() {
+ return {};
+ }
+
virtual lldb::ScriptedPlatformInterfaceUP GetScriptedPlatformInterface() {
return {};
}
@@ -597,6 +603,8 @@ class ScriptInterpreter : public PluginInterface {
lldb::ExecutionContextRefSP GetOpaqueTypeFromSBExecutionContext(
const lldb::SBExecutionContext &exe_ctx) const;
+ lldb::ModuleSP GetOpaqueTypeFromSBModule(const lldb::SBModule &module) const;
+
protected:
Debugger &m_debugger;
lldb::ScriptLanguage m_script_lang;
diff --git a/lldb/include/lldb/Target/Process.h b/lldb/include/lldb/Target/Process.h
index 536a69fb89759..305a682f0ad83 100644
--- a/lldb/include/lldb/Target/Process.h
+++ b/lldb/include/lldb/Target/Process.h
@@ -87,6 +87,7 @@ class ProcessProperties : public Properties {
Args GetExtraStartupCommands() const;
void SetExtraStartupCommands(const Args &args);
FileSpec GetPythonOSPluginPath() const;
+ FileSpec GetPythonJITLoaderPath() const;
uint32_t GetVirtualAddressableBits() const;
void SetVirtualAddressableBits(uint32_t bits);
uint32_t GetHighmemVirtualAddressableBits() const;
diff --git a/lldb/include/lldb/lldb-forward.h b/lldb/include/lldb/lldb-forward.h
index c664d1398f74d..837c734a98059 100644
--- a/lldb/include/lldb/lldb-forward.h
+++ b/lldb/include/lldb/lldb-forward.h
@@ -114,6 +114,7 @@ class Instruction;
class InstructionList;
class InstrumentationRuntime;
class JITLoader;
+class JITLoaderInterface;
class JITLoaderList;
class Language;
class LanguageCategory;
@@ -364,6 +365,7 @@ typedef std::shared_ptr<lldb_private::IOHandler> IOHandlerSP;
typedef std::shared_ptr<lldb_private::IOObject> IOObjectSP;
typedef std::shared_ptr<lldb_private::IRExecutionUnit> IRExecutionUnitSP;
typedef std::shared_ptr<lldb_private::JITLoader> JITLoaderSP;
+typedef std::shared_ptr<lldb_private::JITLoaderInterface> JITLoaderInterfaceSP;
typedef std::unique_ptr<lldb_private::JITLoaderList> JITLoaderListUP;
typedef std::shared_ptr<lldb_private::LanguageRuntime> LanguageRuntimeSP;
typedef std::unique_ptr<lldb_private::SystemRuntime> SystemRuntimeUP;
diff --git a/lldb/source/Interpreter/ScriptInterpreter.cpp b/lldb/source/Interpreter/ScriptInterpreter.cpp
index 63655cc5a50c6..0b12425c1c504 100644
--- a/lldb/source/Interpreter/ScriptInterpreter.cpp
+++ b/lldb/source/Interpreter/ScriptInterpreter.cpp
@@ -130,6 +130,12 @@ ScriptInterpreter::GetOpaqueTypeFromSBExecutionContext(
return exe_ctx.m_exe_ctx_sp;
}
+lldb::ModuleSP
+ScriptInterpreter::GetOpaqueTypeFromSBModule(
+ const lldb::SBModule &module) const {
+ return module.m_opaque_sp;
+}
+
lldb::ScriptLanguage
ScriptInterpreter::StringToLanguage(const llvm::StringRef &language) {
if (language.equals_insensitive(LanguageToString(eScriptLanguageNone)))
diff --git a/lldb/source/Plugins/JITLoader/CMakeLists.txt b/lldb/source/Plugins/JITLoader/CMakeLists.txt
index e52230199109f..aad64f1e7c8bd 100644
--- a/lldb/source/Plugins/JITLoader/CMakeLists.txt
+++ b/lldb/source/Plugins/JITLoader/CMakeLists.txt
@@ -1 +1,2 @@
add_subdirectory(GDB)
+add_subdirectory(Python)
diff --git a/lldb/source/Plugins/JITLoader/Python/CMakeLists.txt b/lldb/source/Plugins/JITLoader/Python/CMakeLists.txt
new file mode 100644
index 0000000000000..a64ade6009a2b
--- /dev/null
+++ b/lldb/source/Plugins/JITLoader/Python/CMakeLists.txt
@@ -0,0 +1,11 @@
+add_lldb_library(lldbPluginJITLoaderPython PLUGIN
+ JITLoaderPython.cpp
+
+ LINK_LIBS
+ lldbCore
+ lldbInterpreter
+ lldbSymbol
+ lldbTarget
+ lldbValueObject
+ lldbPluginProcessUtility
+ )
diff --git a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp
new file mode 100644
index 0000000000000..8981d55fc4661
--- /dev/null
+++ b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp
@@ -0,0 +1,146 @@
+//===-- JITLoaderPython.cpp -----------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Host/Config.h"
+
+#if LLDB_ENABLE_PYTHON
+
+#include "JITLoaderPython.h"
+
+#include "Plugins/Process/Utility/RegisterContextDummy.h"
+#include "Plugins/Process/Utility/RegisterContextMemory.h"
+#include "Plugins/Process/Utility/ThreadMemory.h"
+#include "lldb/Core/Debugger.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Core/PluginManager.h"
+#include "lldb/Interpreter/CommandInterpreter.h"
+#include "lldb/Interpreter/ScriptInterpreter.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/VariableList.h"
+#include "lldb/Target/Process.h"
+#include "lldb/Target/StopInfo.h"
+#include "lldb/Target/Target.h"
+#include "lldb/Target/Thread.h"
+#include "lldb/Target/ThreadList.h"
+#include "lldb/Utility/DataBufferHeap.h"
+#include "lldb/Utility/LLDBLog.h"
+#include "lldb/Utility/RegisterValue.h"
+#include "lldb/Utility/StreamString.h"
+#include "lldb/Utility/StructuredData.h"
+#include "lldb/ValueObject/ValueObjectVariable.h"
+
+#include <memory>
+
+using namespace lldb;
+using namespace lldb_private;
+
+LLDB_PLUGIN_DEFINE(JITLoaderPython)
+
+void JITLoaderPython::Initialize() {
+ PluginManager::RegisterPlugin(GetPluginNameStatic(),
+ GetPluginDescriptionStatic(), CreateInstance,
+ nullptr);
+}
+
+void JITLoaderPython::Terminate() {
+ PluginManager::UnregisterPlugin(CreateInstance);
+}
+
+JITLoaderSP JITLoaderPython::CreateInstance(Process *process, bool force) {
+ // Python JITLoader plug-ins must be requested by name, so force must
+ // be true
+ FileSpec python_os_plugin_spec(process->GetPythonJITLoaderPath());
+ if (python_os_plugin_spec &&
+ FileSystem::Instance().Exists(python_os_plugin_spec)) {
+ std::unique_ptr<JITLoaderPython> os_up(
+ new JITLoaderPython(process, python_os_plugin_spec));
+ if (os_up.get() && os_up->IsValid())
+ return os_up;
+ }
+ return nullptr;
+}
+
+llvm::StringRef JITLoaderPython::GetPluginDescriptionStatic() {
+ return "JIT loader plug-in that implements a JIT loader using a python "
+ "class that implements the necessary JITLoader functionality.";
+}
+
+JITLoaderPython::JITLoaderPython(lldb_private::Process *process,
+ const FileSpec &python_module_path)
+ : JITLoader(process), m_interpreter(nullptr), m_script_object_sp() {
+ if (!process)
+ return;
+ TargetSP target_sp = process->CalculateTarget();
+ if (!target_sp)
+ return;
+ m_interpreter = target_sp->GetDebugger().GetScriptInterpreter();
+ if (!m_interpreter)
+ return;
+
+ std::string os_plugin_class_name(
+ python_module_path.GetFilename().AsCString(""));
+ if (os_plugin_class_name.empty())
+ return;
+
+ LoadScriptOptions options;
+ char python_module_path_cstr[PATH_MAX];
+ python_module_path.GetPath(python_module_path_cstr,
+ sizeof(python_module_path_cstr));
+ Status error;
+ if (!m_interpreter->LoadScriptingModule(python_module_path_cstr, options,
+ error))
+ return;
+
+ // Strip the ".py" extension if there is one
+ size_t py_extension_pos = os_plugin_class_name.rfind(".py");
+ if (py_extension_pos != std::string::npos)
+ os_plugin_class_name.erase(py_extension_pos);
+ // Add ".JITLoaderPlugin" to the module name to get a string like
+ // "modulename.JITLoaderPlugin"
+ os_plugin_class_name += ".JITLoaderPlugin";
+
+ JITLoaderInterfaceSP interface_sp =
+ m_interpreter->CreateJITLoaderInterface();
+ if (!interface_sp)
+ return;
+
+ ExecutionContext exe_ctx(process);
+ auto obj_or_err = interface_sp->CreatePluginObject(
+ os_plugin_class_name, exe_ctx);
+
+ if (!obj_or_err) {
+ llvm::consumeError(obj_or_err.takeError());
+ return;
+ }
+
+ StructuredData::GenericSP owned_script_object_sp = *obj_or_err;
+ if (!owned_script_object_sp->IsValid())
+ return;
+
+ m_script_object_sp = owned_script_object_sp;
+ m_interface_sp = interface_sp;
+}
+
+JITLoaderPython::~JITLoaderPython() = default;
+
+void JITLoaderPython::DidAttach() {
+ if (m_interface_sp)
+ m_interface_sp->DidAttach();
+}
+
+void JITLoaderPython::DidLaunch() {
+ if (m_interface_sp)
+ m_interface_sp->DidLaunch();
+}
+
+void JITLoaderPython::ModulesDidLoad(lldb_private::ModuleList &module_list) {
+ if (m_interface_sp)
+ m_interface_sp->ModulesDidLoad(module_list);
+}
+
+#endif // #if LLDB_ENABLE_PYTHON
diff --git a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h
new file mode 100644
index 0000000000000..87146ad216de3
--- /dev/null
+++ b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h
@@ -0,0 +1,62 @@
+//===-- JITLoaderPython.h ---------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef liblldb_JITLoaderPython_h_
+#define liblldb_JITLoaderPython_h_
+
+#include "lldb/Host/Config.h"
+
+#if LLDB_ENABLE_PYTHON
+
+#include "lldb/Target/JITLoader.h"
+#include "lldb/Utility/StructuredData.h"
+
+namespace lldb_private {
+class ScriptInterpreter;
+}
+
+class JITLoaderPython : public lldb_private::JITLoader {
+public:
+ JITLoaderPython(lldb_private::Process *process,
+ const lldb_private::FileSpec &python_module_path);
+
+ ~JITLoaderPython() override;
+
+ // Static Functions
+ static lldb::JITLoaderSP CreateInstance(lldb_private::Process *process,
+ bool force);
+
+ static void Initialize();
+
+ static void Terminate();
+
+ static llvm::StringRef GetPluginNameStatic() { return "python"; }
+
+ static llvm::StringRef GetPluginDescriptionStatic();
+
+ // lldb_private::PluginInterface Methods
+ llvm::StringRef GetPluginName() override { return GetPluginNameStatic(); }
+
+ // lldb_private::JITLoader Methods
+ void DidAttach() override;
+ void DidLaunch() override;
+ void ModulesDidLoad(lldb_private::ModuleList &module_list) override;
+
+protected:
+ bool IsValid() const {
+ return m_script_object_sp && m_script_object_sp->IsValid();
+ }
+
+ lldb_private::ScriptInterpreter *m_interpreter = nullptr;
+ lldb::JITLoaderInterfaceSP m_interface_sp;
+ lldb_private::StructuredData::GenericSP m_script_object_sp;
+};
+
+#endif // LLDB_ENABLE_PYTHON
+
+#endif // liblldb_JITLoaderPython_h_
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/CMakeLists.txt b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/CMakeLists.txt
index ee5e48ad5cdc3..ed254707ebb00 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/CMakeLists.txt
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/CMakeLists.txt
@@ -20,6 +20,7 @@ if (LLDB_ENABLE_LIBEDIT)
endif()
add_lldb_library(lldbPluginScriptInterpreterPythonInterfaces PLUGIN
+ JITLoaderPythonInterface.cpp
OperatingSystemPythonInterface.cpp
ScriptInterpreterPythonInterfaces.cpp
ScriptedPlatformPythonInterface.cpp
@@ -40,5 +41,3 @@ add_lldb_library(lldbPluginScriptInterpreterPythonInterfaces PLUGIN
LINK_COMPONENTS
Support
)
-
-
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.cpp b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.cpp
new file mode 100644
index 0000000000000..43d77964efa49
--- /dev/null
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.cpp
@@ -0,0 +1,79 @@
+//===-- JITLoaderPythonInterface.cpp --------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Core/ModuleList.h"
+#include "lldb/Core/PluginManager.h"
+#include "lldb/Host/Config.h"
+#include "lldb/Target/ExecutionContext.h"
+#include "lldb/Utility/Log.h"
+#include "lldb/lldb-enumerations.h"
+
+#if LLDB_ENABLE_PYTHON
+
+// clang-format off
+// LLDB Python header must be included first
+#include "../lldb-python.h"
+//clang-format on
+
+#include "../SWIGPythonBridge.h"
+#include "../ScriptInterpreterPythonImpl.h"
+#include "JITLoaderPythonInterface.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+JITLoaderPythonInterface::JITLoaderPythonInterface(
+ ScriptInterpreterPythonImpl &interpreter)
+ : ScriptedPythonInterface(interpreter) {}
+
+llvm::Expected<StructuredData::GenericSP>
+JITLoaderPythonInterface::CreatePluginObject(
+ llvm::StringRef class_name, ExecutionContext &exe_ctx) {
+ return ScriptedPythonInterface::CreatePluginObject(class_name, nullptr,
+ exe_ctx.GetProcessSP());
+}
+
+void JITLoaderPythonInterface::Initialize() {
+ const std::vector<llvm::StringRef> ci_usages = {
+ "settings set target.process.python-jit-loader-plugin-path <script-path>"};
+ const std::vector<llvm::StringRef> api_usages = {};
+ PluginManager::RegisterPlugin(
+ GetPluginNameStatic(), llvm::StringRef("JIT loader python plugin"),
+ CreateInstance, eScriptLanguagePython, {ci_usages, api_usages});
+}
+
+void JITLoaderPythonInterface::Terminate() {
+ PluginManager::UnregisterPlugin(CreateInstance);
+}
+
+//------------------------------------------------------------------------------
+// JITLoader API overrides
+//------------------------------------------------------------------------------
+
+void JITLoaderPythonInterface::DidAttach() {
+ Status error;
+ Dispatch("did_attach", error);
+}
+
+void JITLoaderPythonInterface::DidLaunch() {
+ Status error;
+ Dispatch("did_launch", error);
+}
+
+void JITLoaderPythonInterface::ModulesDidLoad(ModuleList &module_list) {
+ Status error;
+ // There is no SBModuleList, so we need to deliver each module individually
+ // to the python scripts since it uses the LLDB public API.
+ module_list.ForEach([&](const lldb::ModuleSP &module_sp_ref) {
+ lldb::ModuleSP module_sp(module_sp_ref);
+ Dispatch("module_did_load", error, module_sp);
+ return true; // Keep iterating
+ });
+}
+
+#endif
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h
new file mode 100644
index 0000000000000..cb8ef5b8c43ad
--- /dev/null
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h
@@ -0,0 +1,104 @@
+//===-- JITLoaderPythonInterface.h ------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_INTERFACES_JITLOADERPYTHONINTERFACE_H
+#define LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_INTERFACES_JITLOADERPYTHONINTERFACE_H
+
+#include "lldb/Host/Config.h"
+#include "lldb/Interpreter/Interfaces/JITLoaderInterface.h"
+
+#if LLDB_ENABLE_PYTHON
+
+#include "ScriptedThreadPythonInterface.h"
+
+#include <optional>
+
+/// Defines a JITLoader interface for Python.
+///
+/// Users can implement a JIT loader in python by implementing a class that
+/// named "JITLoaderPlugin" in a module. The path to this module is specified
+/// in the settings using:
+///
+/// (lldb) setting set target.process.python-jit-loader-path <path>
+///
+/// When the process starts up it will load this module and call methods on the
+/// python class. The python class must implement the following methods:
+///
+/// #---------------------------------------------------------------------------
+/// # The class must be named "JITLoaderPlugin" in the python file.
+/// #---------------------------------------------------------------------------
+/// class JITLoaderPlugin:
+/// #-----------------------------------------------------------------------
+/// # Construct this object with reference to the process that owns this
+/// # JIT loader.
+/// #-----------------------------------------------------------------------
+/// def __init__(self, process: lldb.SBProce...
[truncated]
|
You can test this locally with the following command:git-clang-format --diff HEAD~1 HEAD --extensions cpp,h -- lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.cpp lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h lldb/test/API/functionalities/jitloader_python/jit.cpp lldb/test/API/functionalities/jitloader_python/main.cpp lldb/include/lldb/API/SBModule.h lldb/include/lldb/Interpreter/ScriptInterpreter.h lldb/include/lldb/Target/Process.h lldb/include/lldb/lldb-forward.h lldb/source/Interpreter/ScriptInterpreter.cpp lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptInterpreterPythonInterfaces.h lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.cpp lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.h lldb/source/Plugins/ScriptInterpreter/Python/SWIGPythonBridge.h lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPythonImpl.h lldb/source/Target/Process.cpp lldb/unittests/ScriptInterpreter/Python/PythonTestSuite.cppView the diff from clang-format here.diff --git a/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h b/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h
index b18d7ae5a..84729a2c2 100644
--- a/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h
+++ b/lldb/include/lldb/Interpreter/Interfaces/JITLoaderInterface.h
@@ -15,15 +15,13 @@
namespace lldb_private {
class JITLoaderInterface : virtual public ScriptedInterface {
public:
-
virtual llvm::Expected<StructuredData::GenericSP>
- CreatePluginObject(llvm::StringRef class_name,
+ CreatePluginObject(llvm::StringRef class_name,
lldb_private::ExecutionContext &exe_ctx) = 0;
virtual void DidAttach() {};
virtual void DidLaunch() {};
virtual void ModulesDidLoad(lldb_private::ModuleList &module_list) {};
-
};
} // namespace lldb_private
diff --git a/lldb/include/lldb/Interpreter/ScriptInterpreter.h b/lldb/include/lldb/Interpreter/ScriptInterpreter.h
index f8885987f..a8027fa24 100644
--- a/lldb/include/lldb/Interpreter/ScriptInterpreter.h
+++ b/lldb/include/lldb/Interpreter/ScriptInterpreter.h
@@ -16,8 +16,8 @@
#include "lldb/API/SBEvent.h"
#include "lldb/API/SBExecutionContext.h"
#include "lldb/API/SBLaunchInfo.h"
-#include "lldb/API/SBModule.h"
#include "lldb/API/SBMemoryRegionInfo.h"
+#include "lldb/API/SBModule.h"
#include "lldb/API/SBStream.h"
#include "lldb/Breakpoint/BreakpointOptions.h"
#include "lldb/Core/PluginInterface.h"
@@ -560,9 +560,7 @@ public:
return {};
}
- virtual lldb::JITLoaderInterfaceSP CreateJITLoaderInterface() {
- return {};
- }
+ virtual lldb::JITLoaderInterfaceSP CreateJITLoaderInterface() { return {}; }
virtual lldb::ScriptedPlatformInterfaceUP GetScriptedPlatformInterface() {
return {};
@@ -602,7 +600,7 @@ public:
const lldb::SBExecutionContext &exe_ctx) const;
lldb::ModuleSP GetOpaqueTypeFromSBModule(const lldb::SBModule &module) const;
-
+
protected:
Debugger &m_debugger;
lldb::ScriptLanguage m_script_lang;
diff --git a/lldb/source/Interpreter/ScriptInterpreter.cpp b/lldb/source/Interpreter/ScriptInterpreter.cpp
index 0b12425c1..c95312a8f 100644
--- a/lldb/source/Interpreter/ScriptInterpreter.cpp
+++ b/lldb/source/Interpreter/ScriptInterpreter.cpp
@@ -130,8 +130,7 @@ ScriptInterpreter::GetOpaqueTypeFromSBExecutionContext(
return exe_ctx.m_exe_ctx_sp;
}
-lldb::ModuleSP
-ScriptInterpreter::GetOpaqueTypeFromSBModule(
+lldb::ModuleSP ScriptInterpreter::GetOpaqueTypeFromSBModule(
const lldb::SBModule &module) const {
return module.m_opaque_sp;
}
diff --git a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp
index 8981d55fc..d01109b14 100644
--- a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp
+++ b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.cpp
@@ -104,14 +104,13 @@ JITLoaderPython::JITLoaderPython(lldb_private::Process *process,
// "modulename.JITLoaderPlugin"
os_plugin_class_name += ".JITLoaderPlugin";
- JITLoaderInterfaceSP interface_sp =
- m_interpreter->CreateJITLoaderInterface();
+ JITLoaderInterfaceSP interface_sp = m_interpreter->CreateJITLoaderInterface();
if (!interface_sp)
return;
ExecutionContext exe_ctx(process);
- auto obj_or_err = interface_sp->CreatePluginObject(
- os_plugin_class_name, exe_ctx);
+ auto obj_or_err =
+ interface_sp->CreatePluginObject(os_plugin_class_name, exe_ctx);
if (!obj_or_err) {
llvm::consumeError(obj_or_err.takeError());
diff --git a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h
index 87146ad21..a86e2531d 100644
--- a/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h
+++ b/lldb/source/Plugins/JITLoader/Python/JITLoaderPython.h
@@ -28,7 +28,7 @@ public:
~JITLoaderPython() override;
// Static Functions
- static lldb::JITLoaderSP CreateInstance(lldb_private::Process *process,
+ static lldb::JITLoaderSP CreateInstance(lldb_private::Process *process,
bool force);
static void Initialize();
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h
index cb8ef5b8c..a087b4823 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/JITLoaderPythonInterface.h
@@ -19,7 +19,7 @@
#include <optional>
/// Defines a JITLoader interface for Python.
-///
+///
/// Users can implement a JIT loader in python by implementing a class that
/// named "JITLoaderPlugin" in a module. The path to this module is specified
/// in the settings using:
@@ -34,7 +34,7 @@
/// #---------------------------------------------------------------------------
/// class JITLoaderPlugin:
/// #-----------------------------------------------------------------------
-/// # Construct this object with reference to the process that owns this
+/// # Construct this object with reference to the process that owns this
/// # JIT loader.
/// #-----------------------------------------------------------------------
/// def __init__(self, process: lldb.SBProcess):
@@ -53,8 +53,8 @@
/// pass
///
/// #-----------------------------------------------------------------------
-/// # Called once for each module that is loaded into the debug sessions.
-/// # This allows clients to search to symbols or references to JIT'ed
+/// # Called once for each module that is loaded into the debug sessions.
+/// # This allows clients to search to symbols or references to JIT'ed
/// # functions in each module as it gets loaded. Note that this function
/// # can be called prior to did_attach() or did_launch() being called as
/// # libraries get loaded during the attach or launch.
@@ -64,30 +64,26 @@
///
namespace lldb_private {
-class JITLoaderPythonInterface
- : virtual public JITLoaderInterface,
- virtual public ScriptedPythonInterface,
- public PluginInterface {
+class JITLoaderPythonInterface : virtual public JITLoaderInterface,
+ virtual public ScriptedPythonInterface,
+ public PluginInterface {
public:
JITLoaderPythonInterface(ScriptInterpreterPythonImpl &interpreter);
llvm::Expected<StructuredData::GenericSP>
- CreatePluginObject(llvm::StringRef class_name,
+ CreatePluginObject(llvm::StringRef class_name,
ExecutionContext &exe_ctx) override;
-
llvm::SmallVector<AbstractMethodRequirement>
GetAbstractMethodRequirements() const override {
return llvm::SmallVector<AbstractMethodRequirement>(
- {{"did_attach"},
- {"did_launch"},
- {"module_did_load", 1}});
+ {{"did_attach"}, {"did_launch"}, {"module_did_load", 1}});
}
-
+
void DidAttach() override;
void DidLaunch() override;
void ModulesDidLoad(lldb_private::ModuleList &module_list) override;
-
+
static void Initialize();
static void Terminate();
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.cpp b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.cpp
index 6d2b95fd7..8291ab58d 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.cpp
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.cpp
@@ -180,12 +180,11 @@ ScriptedPythonInterface::ExtractValueFromPythonObject<
template <>
lldb::ModuleSP
-ScriptedPythonInterface::ExtractValueFromPythonObject<
- lldb::ModuleSP>(python::PythonObject &p, Status &error) {
+ScriptedPythonInterface::ExtractValueFromPythonObject<lldb::ModuleSP>(
+ python::PythonObject &p, Status &error) {
- lldb::SBModule *sb_module =
- reinterpret_cast<lldb::SBModule *>(
- python::LLDBSWIGPython_CastPyObjectToSBModule(p.get()));
+ lldb::SBModule *sb_module = reinterpret_cast<lldb::SBModule *>(
+ python::LLDBSWIGPython_CastPyObjectToSBModule(p.get()));
if (!sb_module) {
error = Status::FromErrorStringWithFormat(
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.h b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.h
index 011cee2ea..7cc21de3f 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.h
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/Interfaces/ScriptedPythonInterface.h
@@ -594,8 +594,8 @@ ScriptedPythonInterface::ExtractValueFromPythonObject<
template <>
lldb::ModuleSP
-ScriptedPythonInterface::ExtractValueFromPythonObject<
- lldb::ModuleSP>(python::PythonObject &p, Status &error);
+ScriptedPythonInterface::ExtractValueFromPythonObject<lldb::ModuleSP>(
+ python::PythonObject &p, Status &error);
} // namespace lldb_private
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp b/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
index 54093765f..83373a889 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
@@ -1569,7 +1569,6 @@ JITLoaderInterfaceSP ScriptInterpreterPythonImpl::CreateJITLoaderInterface() {
return std::make_shared<JITLoaderPythonInterface>(*this);
}
-
StructuredData::ObjectSP
ScriptInterpreterPythonImpl::CreateStructuredDataFromScriptObject(
ScriptObject obj) {
diff --git a/lldb/source/Target/Process.cpp b/lldb/source/Target/Process.cpp
index 5f2dfed22..e7a98606d 100644
--- a/lldb/source/Target/Process.cpp
+++ b/lldb/source/Target/Process.cpp
@@ -213,7 +213,6 @@ FileSpec ProcessProperties::GetPythonJITLoaderPath() const {
return GetPropertyAtIndexAs<FileSpec>(idx, {});
}
-
uint32_t ProcessProperties::GetVirtualAddressableBits() const {
const uint32_t idx = ePropertyVirtualAddressableBits;
return GetPropertyAtIndexAs<uint64_t>(
diff --git a/lldb/test/API/functionalities/jitloader_python/jit.cpp b/lldb/test/API/functionalities/jitloader_python/jit.cpp
index 6a8ec50e6..82e34de0b 100644
--- a/lldb/test/API/functionalities/jitloader_python/jit.cpp
+++ b/lldb/test/API/functionalities/jitloader_python/jit.cpp
@@ -3,42 +3,38 @@
// GDB JIT interface
enum JITAction { JIT_NOACTION, JIT_REGISTER_FN, JIT_UNREGISTER_FN };
-struct JITCodeEntry
-{
- struct JITCodeEntry* next;
- struct JITCodeEntry* prev;
- const char *symfile_addr;
- uint64_t symfile_size;
+struct JITCodeEntry {
+ struct JITCodeEntry *next;
+ struct JITCodeEntry *prev;
+ const char *symfile_addr;
+ uint64_t symfile_size;
};
-struct JITDescriptor
-{
- uint32_t version;
- uint32_t action_flag;
- struct JITCodeEntry* relevant_entry;
- struct JITCodeEntry* first_entry;
+struct JITDescriptor {
+ uint32_t version;
+ uint32_t action_flag;
+ struct JITCodeEntry *relevant_entry;
+ struct JITCodeEntry *first_entry;
};
-struct JITDescriptor __jit_debug_descriptor = { 1, JIT_NOACTION, 0, 0 };
+struct JITDescriptor __jit_debug_descriptor = {1, JIT_NOACTION, 0, 0};
-void __jit_debug_register_code()
-{
-}
+void __jit_debug_register_code() {}
// end GDB JIT interface
struct JITCodeEntry entry;
-int main()
-{
- // Create a code entry with a bogus size
- entry.next = entry.prev = 0;
- entry.symfile_addr = (char *)&entry;
- entry.symfile_size = (uint64_t)47<<32;
+int main() {
+ // Create a code entry with a bogus size
+ entry.next = entry.prev = 0;
+ entry.symfile_addr = (char *)&entry;
+ entry.symfile_size = (uint64_t)47 << 32;
- __jit_debug_descriptor.relevant_entry = __jit_debug_descriptor.first_entry = &entry;
- __jit_debug_descriptor.action_flag = JIT_REGISTER_FN;
+ __jit_debug_descriptor.relevant_entry = __jit_debug_descriptor.first_entry =
+ &entry;
+ __jit_debug_descriptor.action_flag = JIT_REGISTER_FN;
- __jit_debug_register_code();
+ __jit_debug_register_code();
- return 0;
+ return 0;
}
diff --git a/lldb/test/API/functionalities/jitloader_python/main.cpp b/lldb/test/API/functionalities/jitloader_python/main.cpp
index 3b1d5e229..fbca82b35 100644
--- a/lldb/test/API/functionalities/jitloader_python/main.cpp
+++ b/lldb/test/API/functionalities/jitloader_python/main.cpp
@@ -1,48 +1,37 @@
#include <inttypes.h>
#include <stdio.h>
-enum JITAction {
- JIT_NOACTION = 0,
- JIT_LOAD = 1,
- JIT_UNLOAD = 2
-};
+enum JITAction { JIT_NOACTION = 0, JIT_LOAD = 1, JIT_UNLOAD = 2 };
-struct JITEntry
-{
- struct JITEntry* next = nullptr;
- const char *path = nullptr;
- uint64_t address = 0;
+struct JITEntry {
+ struct JITEntry *next = nullptr;
+ const char *path = nullptr;
+ uint64_t address = 0;
};
-
JITEntry *g_entry_list = nullptr;
-void jit_module_action(JITEntry *entry, JITAction action)
-{
- printf("entry = %p, action = %i\n", (void *)entry, action);
+void jit_module_action(JITEntry *entry, JITAction action) {
+ printf("entry = %p, action = %i\n", (void *)entry, action);
}
// end GDB JIT interface
-
-int main()
-{
- // Create an empty JITEntry. The test case will set the path and address
- // with valid values at the first breakpoint. We build a "jit.out" binary
- // in the python test and we will load it at a address, so the test case
- // will calculate the path to the "jit.out" and the address to load it at
- // and set the values with some expressions.
- JITEntry entry;
-
- // Call the "jit_module_action" function to cause our JIT module to be
- // added to our target and loaded at an address.
- jit_module_action(&entry, JIT_LOAD); // Breakpoint 1
- printf("loaded module %s at %16.16" PRIx64 "\n",
- entry.path,
- entry.address);
- // Call the "jit_module_action" function to cause our JIT module to be
- // unloaded at an address.
- jit_module_action(&entry, JIT_UNLOAD); // Breakpoint 2
- printf("unloaded module %s" PRIx64 "\n", // Breakpoint 3
- entry.path);
- return 0;
+int main() {
+ // Create an empty JITEntry. The test case will set the path and address
+ // with valid values at the first breakpoint. We build a "jit.out" binary
+ // in the python test and we will load it at a address, so the test case
+ // will calculate the path to the "jit.out" and the address to load it at
+ // and set the values with some expressions.
+ JITEntry entry;
+
+ // Call the "jit_module_action" function to cause our JIT module to be
+ // added to our target and loaded at an address.
+ jit_module_action(&entry, JIT_LOAD); // Breakpoint 1
+ printf("loaded module %s at %16.16" PRIx64 "\n", entry.path, entry.address);
+ // Call the "jit_module_action" function to cause our JIT module to be
+ // unloaded at an address.
+ jit_module_action(&entry, JIT_UNLOAD); // Breakpoint 2
+ printf("unloaded module %s" PRIx64 "\n", // Breakpoint 3
+ entry.path);
+ return 0;
}
|
|
This seems like a generic module loading observer. I don't see anything JIT specific about it. Not saying a generic module loading observer is not a good idea. But calling it a JITLoader seems pretty confusing to me. |
|
This also seems like an awkward way to do what we've wanted for a while, which is the equivalent of stop hooks for "launch", "attach" and "module loaded", since you have to do all three, even if you only wanted to do one of the set. |
|
I changed the stop-hooks recently so they optionally fire when lldb first gets control of the process, so you can already write python code that intervenes when your "did_attach" and "did_launch" callbacks fire. So only the module_loaded callback is new. |
|
One thing that's nicer about your approach here over independent callbacks for each of the hooks is that it allows you to group the three callbacks in the same class instance, so that you can more easily keep shared state. But that's a general problem with these affordances. For instance, it's super inconvenient that the summary providers and synthetic child providers produce separate objects to handle the callbacks. You end up computing the sizes of container classes twice, once for the summary and once because you need it to present the type... If the summary & child provider could share state, you'd only need to do that once per stop. And as we are adding more of these callbacks for "lifecycle events" it would be really convenient, as you have done here, to be able to get one object to manage multiple different types of callback hits. For the case of hooks, I wonder if we could augment the -C options we currently use to add affordances backed by a Python class with a
and then to finish off your design, we'd add:
The --shared-instance that would tell lldb to make a single object instance and reuse if for any --shared-instance addition that uses this class (one for each target in this case). Allowing you to specify the method name means you don't have to do the discrimination in That way as we add more of these callbacks (which I agree we really need to do) people could mix and match them as the wish, and we wouldn't have to figure out the right combination(s) for everybody's use cases. Note for summary & synthetic child providers you wouldn't need to specify the methods as those are pre-determined. So the It might also be handy to be able to define several commands that share state, so you could do the same thing for command classes (though in that case the shared instance would be held by the debugger not the target...) |
|
But if you don't care so much about shared state, then I think a better way of doing what you want is just to add |
medismailben
left a comment
medismailben
left a comment
There was a problem hiding this comment.
Hey @clayborg, this is pretty cool. I'm glad you were able to use and extend the ScriptedPythonInterface to implement this, hopefully it wasn't too complicated. LGTM!
vogelsgesang
changed the title
+1 on the name. The design seems very general, so maybe calling it something like ModuleObserver or something would make more sense. Your PR summary describes what users can do, but why might they want to do it? Do you have a motivation for this change? |
I meant LGTM on the scripting side of things. I still thing we should address Jim's comments:
If you use a I like the idea of being able to create or share instances between scripted affordances but I think we need a way to list the existing instances (and what scripted affordance are they driving), otherwise, how would the user be able to tell what number to pass to Suggestion: if you make this more generic, you could implement a template that's included in the lldb python module like we do for other scripted affordances (ScriptedProcess, ScriptedThread, ScriptedThreadPlan ...) with a base implementation for every function the script should implement. That would make it easier to write new scripts for this. |
I do, and I should have mentioned this in the description: I want to extend the JITLoader API to be able to lazily resolve addresses. Right now JIT loaders are inefficient and expensive to use as they must stop anytime something has been JIT'ed and if a program JITs a million functions, we will stop a million times due to it setting a breakpoint in the process. I changed it so that any "load address -> lldb_private::Address" resolving goes though With this in, we can modify the This allows JIT loaders to be loaded with little or no cost until we do a backtrace. Any frames that go through JIT'ed functions that are not resolved allows the JIT loaded to load only what is needed. Another future patch will notify the JITLoader plug-ins about breakpoints as they are created and allow the JIT loader to intelligently implement functionality to set breakpoints. We have an internal JIT solution that already does this. When a user sets a file and line breakpoint, or a breakpoint by name, the JIT loader will mark up its metadata and stop in the debugger when the code for this file and line breakpoint or breakpoint by name gets JIT'ed. Again, this helps us debug processes that JIT without the huge overhead that the current model imposes. This patch lays the groundwork for current JITLoader plug-ins in python, and will allow me to modify the JITLoader API and test the new changes in python. It will allow me to break up the new changes piece by piece:
@jimingham Hopefully this explains why this patch exists. I will read your comments above and comment if needed. |
true, in the current form. Check out the comments in my previous reply about the motivations for this.
It would be great to create a generic version of this that allows users to add methods to a python class and just have it be notified. That being said, by motivation here is to augment the JITLoader plug-in API to do more things as mentioned above. Now we can probably do this with a notification class that we can install as long as we can find out about everything that is needed. So one approach might be to allow a python class to get all of the notifications I need for the JITLoader plug-in without calling it a JITLoader as you wanted. The class could be something like: We could remove the JITLoader class all together. |
I think I'd come at if from the opposite direction. We don't currently know what the full set of messages that we want to send are, so making one class that receives all the messages we know about at present seems limiting. What I was proposing instead is that when we add a way to register a callback to some event in lldb, we extend the registry to indicate not just the class that will be instantiated to watch the event, but which method is the responder. That way, for instance, you can register a stop-hook with your class, and then you will have launch and attach callbacks already. But you need a way to say "Use a common instance of this class per-whatever entity owns that callback" and "use this method(s) on my object". That way we don't have to hook everything up for you, but rather it will be easy for designers to make a class where they can hook up the particular callbacks they need. |
|
It also seems architecturally wrong to try to guess and influence what BreakpointResolvers do behind their backs. After all, the resolver might be just some Python Code you know nothing about. How would you instrument that? If I set a regular expression name breakpoint, will you know to compare that regex against what the JIT produces? What about source regular expression breakpoints? Do you figure out what the containing source file is and observe that? Having a system where "if you set these kinds of breakpoints we'll be able to intervene, but other breakpoint types just won't work" seems awkward. If you are going to only support certain breakpoint types for JIT debugging, it seems much better to make that an explicit JIT breakpoint type writing a custom resolver that cooperates with your JIT engine to register interest and get called back when JIT events occur that are relevant to it. Either that or we need to introduce the notion of a "dynamic symbol resolver" that you can register information about file names or symbol names, and then have the standard breakpoint resolvers check if one of these exists and registers interest for the names and files it is looking for. But trying to suss out what a resolver is going to do from the outside isn't the right way to go. |
We currently only handle source file + line breakpoints and breakpoints by name. For source file + line breakpoints the JIT keeps metadata that says "this function contains these source file + line ranges". When we get notified that a breakpoint was set, we just need to know the source file + line, and then it allows the JIT to modify the metadata it contains if the function hasn't been JIT'ed yet. If it has been JIT'ed, it will load the debug info for that function immediately if it hasn't already been loaded. This allows the breakpoint to naturally resolve itself as soon as the debug info is loaded. If the debug info hasn't been loaded, then the modification to the metadata in the JIT marks the function as being needed by the debugger and if and when and only when it gets JIT'ed we will load the module for it and the breakpoint will naturally resolve itself. Same thing for functions by name.
We have a system that is already working just being notified about breakpoints in the JIT loader. Yes, it doesn't handle all breakpoint types right now, but we are getting this to work as proof of concept with a JIT loader that does everything lazilly. So right now only functions that have breakpoints set in them need debug info to be generated and that is if and only if they ever get JIT'ed. It works quite well, abeit we only support two kinds of breakpoints currently. Then if a stack trace goes through a function that doesn't resolve, we can lazily load the debug info for it on the fly only when we have a backtrace that traverses through a JIT'ed frame that we don't have debug info for yet.
Happy to meet and discuss anytime. But this PR has isn't doing any of those things yet. This just enabled python JIT loaders which we need for other purposes as well.
How do we store an instance of a class and then call a method on it? We have no notion of a baton in python callbacks right now. If we allowed python callbacks to be registered with any python object as a baton, then this could be made to work, but we probably shouldn't try to call a method on some object as there is no way to specify that on the command line. We should also be able to do this with an API in the public API via callbacks with batons.
That is what I don't know how to implement. How would we do this on the command line? Would we need a global variable to contain the class instance?
I am fine with this as long as the solution doesn't require using command line commands to do it and we have APIs. How about public APIs where we register callbacks and make sure that when doing it through python we can specify a python object as a baton that gets given back when the callback is called. Right now all batons for python are non existent because we use the native baton for the python implementation. |
|
We don't have SB API's currently for the stop-hooks yet, so we're free to invent them. But my notion was something like this: Then for stop hooks that have a lot of "when to stop" as opposed to "what to do when you stop" options: Then an opaque class that we will use to find the instance of the class that backs this particular hook: Then in Target, we'd add: Internally, lldb will do what is currently already does, instantiate an instance of script_class_name to handle this hook stash that away somewhere. But we'll add a return that allows us to look up the implementation object, that's the HookInstance class. Then, when we hit a stop for this target, we go through the stop hook instances and dispatches to the method in hook_method_name. This is something we also already know how to do (at least for the Python scripting language), we just hard-coding the name to If we had another hook, like ModulesLoaded, then we'd do: I don't know whether we'll end up needing separate "HookOptions" class, but the stop hooks can filter in a lot of ways that wouldn't make sense for module loaded, so it's likely good to separate those out. Again, internally lldb would do the same thing it currently does for Target Stop Hooks, it would make an instance (held in the Target) of the python class and when modules were loaded, would call that instance's hook_method_name. Then the point of the HookInstance is that we'd add to the HookDefinition:
We also need to handle init methods in the case where we are sharing an instance, because you might for instance want to pass different information to set up the various hooks (e.g. different extra_args). We could do that by requiring that the implementation classes have trivial This scheme seems doable in a language that has runtime dispatch to me. The lldb_private API layers (and their SWIG-ed equivalents) don't need to pass anything but strings: script class name, and method name, and potentially this opaque HookInstance, and lldb will manage the implementation object(s) and dispatching to them - which it already knows how to do because that's how python stop hooks work. That way as we add more kinds of hooks, you can either choose to have each hook managed by an independent instance of your provided class, or have one instance that gets a bunch of hooks routed to it so it can aggregate data among the hooks. That would then allow you to create the composite hook handler that you want. I'm not sure how you would provide a C++ way to do this, but maybe that's not so important. Or maybe someone better at esoteric C++ than me can think of a way to do that? On the command-line side, we already return a hook ID when you do
You seemed to have a much more complicated picture, and I wasn't sure how batons were getting involved. Maybe I'm missing some subtlety? But this scheme seems pretty straightforward. |
|
To do what you want you'd also have to add: class Target { and you can have at it. I have no particular objection to code that's sitting not in lldb trying to figure out what breakpoint resolvers are doing from the outside. I wouldn't want that code in lldb proper because it's bound to fail in many cases, and that doesn't seem appropriate for an implementation in lldb proper. If we're going to do that then we need a more callback based approach at the SymbolContext lookup level that the resolvers can all call when they run. |
5 participants
Users can implement a JIT loader in python by implementing a class that named "JITLoaderPlugin" in a module. The path to this module is specified in the settings using:
When the process starts up it will load this module and call methods on the python class. The python class must implement the following methods: